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SSIP Phase II Process Guide 


The online version of this guide is available at: http://ectacenter.org/topics/ssip/ssip_phase2.asp 


The Part C State Performance Plan (SPP) and Annual Performance Report (APR): Part C Indicator 
Measurement Table describes the focus of the SSIP Phase II—Plan as: 


“The focus of Phase II is on building state capacity to support Early Intervention Service (EIS) programs 
and/or EIS providers with the implementation of evidence-based practices (EBPs) that will lead to 
measurable improvement in the State-identified Measurable Result(s) (SIMR) for infants and toddlers 
with disabilities and their families.” 


The Part C State Performance Plan (SPP) and Annual Performance Report (APR): Part C Indicator 
Measurement Table (subsequently referred to as the Part C Indicator Measurement Table) defines three 
components which must be included in the April 2016 submission of the SPP/APR Indicator C-11 and B-17, 
the State Systemic Improvement Plan (SSIP) including: 

1. Infrastructure Development; 


2. Support for EIS Programs and/or EIS Provider Implementation of Evidence-Based Practices; and 


3. Evaluation 


The SSIP Phase II Process Guide describes the steps needed to accomplish the work of the three 
components of Phase II. Thus, the SSIP Phase II Process Guide is organized according to the following 
sequential phases of work to be done, rather than by the three components. 

1. Kicking off Phase II 


2. Developing the improvement plan 
3. Developing the evaluation plan 


4. Communicating the plan 


Throughout The SSIP Phase II Process Guide the term “the plan” refers to the set of activities, steps, and 
resources described in the Part C Indicator Measurement Table. 


The three components are included throughout the sections of the SSIP Phase II Process Guide. 
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Figure 1: The components included in Phase | and Phase Il of the SSIP 
and the connection between the Phases 


Phase II SSIP Improvement Plan | 


improvement Plan 


- Implement coherent improvement strategies: 
Goals, activities, steps with timelines, 
resources and who's responsible to 
improve insfrastructure and support locals - Data collection/analysis methods & timelines 
in implementing evidence-based practices to measure implementation (process) & 

- Align, partner & leverage existing multiple outcomes (impact) 

Offices, initiatives and other resources - Stakeholder engagement and communication 


Evaluation Plan 
- Evaluation aligned to TOA 


- Short- and long-term outcomes aligned to 
Implementation Plan 


Theory of Action 


5.E,a 


State-Indentified Measureable Child and/or Family Result 


In-Depth Data Analysis In-Depth Infrastructure Assessment 


Primary Concern(s) / Focus(es) 


Broad Data Analysis Broad Infrastructure Assessment 


Getting Started / Preparation 


Figure 1 shows the link between Phase I—Analysis and Phase II-Plan and the connections among the 
components of Phase II. Phase II should be integrally connected to Phase I—Analysis and Phase 
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III-Implementation and Evaluation. The theory of action (ToA) developed in Phase | will drive Phase II and 
lay out the work to be completed in Phase III. Many states are taking steps to install some of the system 
changes, like enhancements to the professional development system, that are most critical to supporting 
the improvement strategies identified in the SSIP. Although these implementation activities are required 
under Phase III and not Phase II, beginning to install them early in the process may be critical to meeting 
future timelines. States also are likely to make changes to the Phase | ToA and improvement strategies, in 
order to align the work to the most recent knowledge of state strengths, challenges, and capacity, and they 
will need to document the reasons for these changes. As states begin the installation of activities to 
improve the infrastructure in Phase II, they should track the activities, including the start date and progress, 
to ensure that they can accurately report and evaluate all activities they completed under the SSIP. 


meine ter ounmissionie Orr. 


Figure 2 shows the timeline for submission of each Phase of the SSIP to OSEP and provides a brief 
description of the content included in that Phase. 


Figure 2: Timelines and Descriptions of Each Phase of the SSIP, 
adapted from the Part C Indicator Measurement Table 


Year 1 — FFY 2013 
Delivered April 2015 


e Data Analysis 


e Description of State 
Infrastructure to Support 
Improvement and Build 
Capacity 


e State-identified 
Measureable Result 
(SIMR) 


e Selection of Coherent 
Improvement Strategies 


e Theory of Action 


Year 2 - FFY 2014 
Due April 2016 


Phase II: Plan 


Multi-year plan addressing: 
e Phase | Content/Updates 


e Infrastructure 
Development 


e Support for EIS Program 
and/or EIS providers in 
Implementing 
Evidence-Based Practices 


e Evaluation Plan 


Years 3-6 - FFY 2015-18 
Due February 2017- 2020 


Phase III: Implementation 
and Evaluation 


Reporting on Progress 


including: 
e Phase | and Phase II 
Content/Updates 


e Progress toward short- 
and long-term outcomes 


e Revisions to the SPP and 
evaluation data to support 
decision 


States will submit Phase II SSIPs through GRADS360 using a process similar to that used for Phase | 


according to OSEP requirements. 
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In the Part C Indicator Measurement Table OSEP stressed the importance of stakeholder engagement 
throughout the SSIP process. Some considerations related to engaging stakeholders in kicking off the 
Phase II plan include: 
e Reengage stakeholders to the work of Phase II based on the status of Phase | of the SSIP and the 
purpose of the SSIP. 


Use multiple opportunities and formats (e.g. websites, newsletter, state conferences) to share 
information about the development of Phase II of the SSIP with stakeholders and create opportunities 
with stakeholders so they can share information with their communities. 


Consider the "messaging" for the participating local programs. Are the messages co-created? How will 
the state garner their support? If the state selected subsets of programs having low performance to 
achieve the SIMR, how will the state garner programs' support without focusing on the negatives that 
prompted their inclusion in the state’s efforts? 


If the state uses planning or implementation teams, engage stakeholders not included on these teams 
in a discussion about how they would like to get information, provide input into planning, and discuss 
potential implications for future learning. 
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What States Need to Submit as a part of SSIP Phase Il 


| Excerpted from: Part C State Systemic Improvement Plan (SSIP) Phase Il OSEP Guidance and Review Tool. 


Updated Data 


In its FFY 2014 through FFY 2018 SPPs/APRs, due yearly February 2016 through February 2020, the 
state must provide updated data for the specific FFY (expressed as percentages) and those data must be 
aligned with the SIMR(s) for Infants and Toddlers with Disabilities and their Families. In its FFY 2014 
through FFY 2018 SPPs/APRs, the state must report on whether or not it met its target. 

Part C SPP/APR Collection Tool 


Part C Indicator Measurement Table 


Components 


Phase II Component #1: Infrastructure Development 
1a) Specify improvements that will be made to the state infrastructure to better support EIS programs 
and providers to implement and scale up EBPs to improve results for infants and toddlers with 


disabilities and their families. 

1b) Identify the steps the state will take to further align and leverage current improvement plans and 
other early learning initiatives and programs in the state, including Race to the Top-Early Learning 
Challenge, Home Visiting Program, Early Head Start, and others which impact infants and 
toddlers with disabilities and their families. 

1c) Identify who will be in charge of implementing the changes to infrastructure, resources needed, 
expected outcomes, and timelines for completing improvement efforts. 

1d) Specify how the state will involve multiple offices within the State Lead Agency, as well as other 
state agencies and stakeholders in the improvement of its infrastructure. 
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Phase II Component #2: Support for EIS programs and providers Implementation of 
Evidence-Based Practices 
2a) Specify how the state will support EIS providers in implementing the evidence-based practices that 
will result in changes in Lead Agency, EIS program, and EIS provider practices to achieve the 
SIMR(s) for infants and toddlers with disabilities and their families. 


2b) Identify steps and specific activities needed to implement the coherent improvement strategies, 
including communication strategies and stakeholder involvement; how identified barriers will be 
addressed; who will be in charge of implementing; how the activities will be implemented with 
fidelity; the resources that will be used to implement them; and timelines for completion. 


2c) Specify how the state will involve multiple offices within the Lead Agency (and other state agencies 
such as the SEA) to support EIS providers in scaling up and sustaining the implementation of the 
evidence-based practices once they have been implemented with fidelity. 


Phase II Component #3: Evaluation 
3a) Specify how the evaluation is aligned to the theory of action and other components of the SSIP 
and the extent to which it includes short-term and long-term objectives to measure implementation 
of the SSIP and its impact on achieving measurable improvement in SIMR(s) for infants and 
toddlers with disabilities and their families. 


3b) Specify how the evaluation includes stakeholders and how information from the evaluation will be 
disseminated to stakeholders. 


3c) Specify the methods that the state will use to collect and analyze data to evaluate implementation 
and outcomes of the SSIP and the progress toward achieving intended improvements in the 
SIMR(s). 

3d) Specify how the state will use the evaluation data to examine the effectiveness of the 
implementation; assess the state’s progress toward achieving intended improvements; and to 
make modifications to the SSIP as necessary. 


Phase II Technical Assistance and Support 


Describe the support the state needs to develop and implement an effective SSIP. 
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Kicking Off SSIP Phase II 


The first step in getting started on Phase II is identifying who will do the work, what they will do, and when 
and how they will do the work. Planning up-front facilitates clear communication with state staff and 
stakeholders about the types of work they will be expected to do and the time commitment they are 
making. States can chose to complete the work through a number of structures, one of which is to develop 
or use existing teams to plan and oversee implementation. As teams are created it is important for states to 
be intentional about how stakeholders are actively involved in the work. If teams are used to complete the 
work of Phase II, defining the purpose and membership of the team and processes they will follow will 
make the teams more efficient and avoid miscommunication. Four key concepts from Implementation 
Science will be integrated into this part of the SSIP Phase II Process Guide including: State Leadership 
Team, Implementation Team, Feedback Loops, and Terms of Reference. 


Step 1. Review and address OSEP's recommendations for improving Phase I. 


Purpose Resources 


e Ensure work is aligned to OSEP's Feedback from OSEP state contact. 
expectations for the state. 


Step 2. Identify the activities and timelines described in Phase | that need to be completed 
during Phase Il. 


Purpose Resources 


e Ensure that sufficient progress is being There are no resources listed for this step. 
made on the activities identified in Phase I. 
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Step 3. Review the requirements for Phase Il. 


Purpose 


e Align planning and deliverables to the 
requirements OSEP described. 


Resources 


Part C SPP/APR Indicator Measurement Table 
Part B SPP/APR Indicator Measurement Table 


Part C State Systemic Improvement Plan (SSIP) 
Phase II OSEP Guidance and Review Tool 


Part B State Systemic Improvement Plan (SSIP) 
Phase II OSEP Guidance and Review Tool 


Step 4. Generate an initial Gantt Chart listing the work to be done in Phase Il. 


Purpose 


e Create a visual of the activities and 
timelines required to complete Phase II. 


e Have a tool to track state progress toward 
completing Phase II. 


Resources 


SSIP Phase II Gantt Chart Template 


Step 5. Identify a staffing structure and those responsible for completing Phase I. 


Purpose 


e Identify the team or teams that will be 
responsible for completing Phase II of the 
SSIP (State Leadership Team, 
Implementation Teams). 


e Complete initial planning of how those 
teams will coordinate (Linked Teams). 


e Identify staff with expertise and interest to 
support the evaluation design and make 
sure the team or teams that are completing 
the planning have access to these staff. 


Resources 


A Guide to the Implementation Process: Stages, 
Steps, and Activities: Stage 2: Installation 


Al Hub Module 3: Implementation teams 


An Integrated Stage-Based Framework for 
Implementation of Early Childhood 
Programs and Systems 


Planning Guide to Statewide Implementation, 
Scale-up, and Sustainability of 
Recommended Practices 


State Leadership Team Benchmarks of Quality 
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Step 6. Describe the role of stakeholders in Phase I. 


Purpose Resources 
e Ensure that various viewpoints and Leading by Convening: A Blueprint for Authentic 
expertise will have a role throughout the Engagement 


development of Phase II. Circles of Involvement 


e Identify the people who need to be invited to 
participate. 


e Ensure that stakeholders are involved in 
creating the activities and undertaking 
relevant pieces of the plan. 


Step 7. If state uses a State Leadership Team and Local Implementation Teams, invite team 
members to participate in Phase Il. 


Purpose Resources 


e Alert potential team members to the need State Leadership Team Benchmarks of Quality 
for a time commitment. 


e Establish the group that will need to be 
moved through orientation to Phase Il. 
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Step 8. Ensure stakeholders and planning team members have an active role in Phase Il. 


Purpose Resources 


e Increase stakeholder understanding of the Getting Ready for Phase II of the SSIP 
findings and conclusions of Phase I. 


e Increase the active role of stakeholders for 
Phase II. 


e Increase understanding of the requirements 
of Phase Il. 


e Increase focus of stakeholder discussion on 
the key requirements of Phase Il. 


e Increase stakeholder engagement in 
actively planning for Phase Il. 


e Have an expectation that stakeholders will 
share these ideas with their networks and 
bring back relevant ideas. 
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Developing the Improvement Plan 


"Planning is what you do before you do something, so that when you do it, it is not all mixed up." 


~ Christopher Robin to Winnie the Pooh (A.A. Milne) 


This section describes the steps in developing an improvement plan. While there are many ways that a 
state may choose to develop a written improvement plan, this guide provides the basic steps in 
improvement planning. A state may complete the steps in a different order, or add or eliminate steps and 
should use this resource in a way that fits state needs. 


During Phase | of the SSIP, states reviewed their data at all levels, assessed their current infrastructure, 
and developed Coherent Improvement Strategies and a Theory of Action that specifies how the State- 
Identified Measureable Result (SIMR) will be achieved. Now it is time to put this all into action! An 
improvement plan for Phase II will provide the details needed to achieve the expected outcomes and make 
progress toward achieving the state's SIMR. 

Improvement planning is an organizational management activity that is used to: 


e Set priorities 
e Focus energy and resources 
e Ensure that internal and external stakeholders are working toward common goals 


e Establish agreement on intended outcomes/results implementation 


Planning occurs all the time, both informally and formally. Effective improvement planning is intentional and 
provides an opportunity to assess the current status, determine if what is being done needs to be done, 
and decide where to go in the future. Improvement planning involves setting priorities, assessing internal 
and external resources, and engaging all interested parties in defining the activities, desired outcomes, and 
implementing/revising improvement strategies as needed to achieve the SIMR. 


In Phase II, developing the written plan will provide states with the structure and detail needed to achieve 
the SIMR through implementation of the improvement strategies. Including stakeholders, key partners, and 
staff in this process ensures that all perspectives are represented. The evaluation measures provide the 
feedback needed to track progress and make needed adjustments. 
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The improvement plan must address improving the infrastructure and how the state will support EIS 
programs and/or EIS providers in implementing the evidence-based practices. It should include the 
following: 

e Activities and steps that will be implemented, who is responsible, and according to what timelines 


e Resources needed for each activity 

e Leverage points and partners from within the lead agency and other initiatives and agencies 
e Communication strategies and stakeholder involvement 

e How identified barriers in the infrastructure will be addressed 


e How activities will be implemented with fidelity 


e Procedures for monitoring the plan's implementation and strategies, timeframe for plan evaluation, and 


modifications, if needed, to ensure progress toward desired outcomes is achieved. 


Improvement Plans answer the following questions: 


e Where are we now? Use this opportunity to review the current environment and clarify the vison, 
mission, and purpose. 


e Where are we going? Think ahead several years to assure consistency and alignment with the 
direction the organization is headed. 


e What will achievement of the SIMR look like? Identify the expected short, intermediate, and 
long-term outcomes that will result from achieving the SIMR. 


e How will we get there? Lay out the road to connect current activities with future desired goals. 
Develop an improvement plan with clearly defined activities. 


e How will we know we've been successful? Determine benchmarks of success. Develop an 


evaluation plan that leads to improving and adjusting the activities to ensure a dynamic and responsive 


process is in place to achieve the SIMR and the intended outcomes. 
Other questions to consider include the following: 
e What actions or changes will occur? 
e Who will carry out these changes? 
e When will the changes will take place, and for how long? 
e What resources are needed to carry out the activity (e.g., staff, funding, supplies)? 


e How will the improvement plan be communicated to stakeholders, the public, providers, and 
families? 


e How will input be provided to make needed adjustments to the improvement plan? 
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Step 1. Convene the core staff and/or stakeholders responsible for the written plan. Core 
Staff/stakeholders responsible for the written improvement plan might represent various levels 
of the system and offer expertise in a broad range of areas. 


Purpose Resources 
e Develop a process for collecting and Planning Guide to Statewide Implementation, 
compiling information for the improvement Scale-up, and Sustainability of 
plan. Recommended Practices 
e Identify fiscal and human resources An Integrated Stage-Based Framework for 
available for planning and/or implementation Implementation of Early Childhood 
of the SSIP. Programs and Systems (see p. 20) 


DIY Committee Guide: Strategic Plan Step 5: 
Writing Your Plan 


Step 2. Determine timeline and responsibilities for developing the written improvement plan. 


Purpose Resources 


e Develop a timeline to ensure submission to Al Hub Activity 5.2: PDSA Who am |? 


OSEF by Apal tl: SSIP Phase II Gantt Chart Template 


e Assign responsibility for completing 
assignments. 


Step 3. Establish the process for developing the improvement plan, including meeting schedule, 
agenda format, and a format for documenting and sharing decisions made in planning 
meetings. 


Purpose Resources 
e Ensure that information provided to planning Community Tool Box: Chapter 8, Section 5: 
groups is consistent. Developing an Action Plan 


e Ensure that information needed for the 
written improvement plan is collected. 
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Step 4. Determine how stakeholders, staff, and partners will be engaged and organized to 
provide input for the improvement plan. 


Purpose Resources 
e Ensure that stakeholders, staff, and partners Leading by Convening: Ensuring Relevant 
actively engage in developing the Participation 


improvement plan. 


e Provide clear expectations on how 
improvement plan will be developed. 


e Assign responsibility for completing 
assignments. 


Step 5. Determine communication protocols to coordinate communication (for the internal 
group actively engaged in developing the improvement plan) at all levels during the planning 
process. 


Purpose Resources 


e Ensure that communication occurs across Al Hub Handout 8: Communication Protocol 
all levels. Worksheet 


e Ensure that all members working on 
developing the improvement plan are in 
communication loop. 


Step 6. Establish protocols for communicating with key external stakeholders who are not 
actively engaged in developing the written plan. 


Purpose Resources 
e Ensure members of the internal planning Template for Strategic Communications Plan 
team communicate about the improvement (see Step 3: Develop Messages) 


plan with other stakeholders in their agency. 


e Ensure buy-in for implementation of the 
SSIP. 
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Step 7. Review the state's or program's vision, mission, and purpose, if appropriate. 


Purpose Resources 


e Ensure alignment of SSIP with state or Vision and Direction in Leadership Checklist 
agency priorities. 


Step 8. Provide brief overview of the Theory of Action (TOA) and Improvement Strategies 
developed in Phase I. This may need to be repeated at each meeting. 


Purpose Resources 


e Ensure planning team members are familiar State Theory of Action 


with the TOA and Improvement Strategies. State Improvement Strategies 


e Keep planners focused on how their work 
fits into the larger SSIP. 


State logic model, if developed 


Step 9. Review and identify evidence-based programs, practices, or approaches that would be 
expected to positively impact the SIMR. This review will yield a set of practices, potential 
programs, or approaches and infrastructure improvements to be considered by planners. 


Purpose Resources 


e Ensure program, practices, or approaches DEC Recommended Practices & Glossary 


align with TO. What Works Clearinghouse 


e Define the set of evidence- based programs, A System Framework for Building High-Quality 


practices, or approaches to be considered Early Intervention and Preschool Special 


by planning teams. Education Programs 
e Define the activities to be completed to 
strengthen the infrastructure for improving 


practices, programs, or approaches. 
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Step 10. Review, discuss, and select potential practices or programs in relation to need, fit, 
resources—sustainability, strength of evidence, readiness for replication and capacity to 
implement. 


Purpose Resources 


e Ensure that the practices selected are The Hexagon Tool 


doable and a good fit. Offering our Best to Children and Families: 


Program-Wide Implementation 


Step 11. Determine the initial implementation sites for the installation of the evidence-based 
program or practice. 


Purpose Resources 


e Ensure that the initial sites are ready for The Hexagon Tool 
installation. 


e Ensure actions are included in the plan 
addressing capacity of installation sites. 


Step 12. Identify short-term and intermediate outcomes that will need to be achieved to improve 
the long-term outcome (SIMR). 


Purpose Resources 
e Ensure the improvement plan activities and Recommended Resources for Planning to 
steps are designed to achieve outcomes of Evaluate Program Improvement Efforts 
the SSIP. (including the SSIP) 


e Link the improvement plan with the 
evaluation plan. 
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Step 13. Select the format to be used to develop the written implementation plan (e.g., sample 
template, template of state choice). 


Purpose Resources 


e Provide document for capturing information Sample SSIP Action Plan Template 
throughout the process of developing the W.K. Kellogg Foundation Logic Model 


written improvement plan. Development Guide 


Step 14. Develop the written improvement plan that identifies how the improvement strategies 
will be implemented to achieve the short-term, intermediate, and long-term outcomes. 


Purpose Resources 
e Ensure written improvement plan follows the Part C SPP/APR Indicator Measurement Table 
Guidance Aan ODEP Ter RNase th Part B SPP/APR Indicator Measurement Table 


¢ Guide implementation of the SSIP work now Part C State Systemic Improvement Plan (SSIP) 
and in future. Phase II OSEP Guidance and Review Tool 
Building an SSIP Evaluation Plan 


Developing a High Quality Improvement Plan 
(see slides 3-11) 


Step 15. Review the written plan to ensure that the activities are Specific, Measureable, 
Achievable, Relevant, and Timely (SMART). 


Purpose Resources 


e Ensure that the activities planned can be Benchmarks of Quality for Home-Visiting 
evaluated. Programs 


Step 16. Share the written plan with stakeholders, parents, providers, agency staff, and partners 
for their review and comment. 


Purpose Resources 


e Ensure buy-in of staff, stakeholders, and There are no resources listed for this step. 
partners. 
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Step 17. Finalize the written plan. 


Purpose Resources 


e Include a narrative summary of the Phase Il There are no resources listed for this step. 
development process and detailed 
improvement plan. 


eae icles tea lied pntonecnt cease reseonreettealy Tegner Sokal temncrsemas 


e Planning may take several sessions. Develop drafts for review and input from stakeholders at 
multiple points in the process. 


e An effective improvement strategy may require several actions and related activities to achieve 
full implementation. 


e Remember to address infrastructure. Identify how you will build on the strengths and address the 
weaknesses identified in Phase | of SSIP development. 


e Carefully consider the evidenced-based practices or approaches to be selected. What practices 
or approaches will most effectively improve outcomes for children and families? 


e Consider evaluation and ways to measure change when developing the improvement plan. How 
will you know activities are implemented with fidelity? How will you know the activities are making the 
expected difference? 


e Criteria for a good improvement plan: 
o Aspirational: Does the improvement plan reflect the current work? Are newly emerging 
opportunities and barriers addressed? Does it address the identified infrastructure issues and 
provide supports for implementation of evidenced-based practices? 


o Complete: Does it list all of the action steps or changes to be sought at all levels of system (e.g., 
practitioner, district, local program, and state)? Does it include partnerships with other programs 
and agencies and leverage existing initiatives? 


o Clearly define responsibilities and timelines: Is it apparent who will do what, by when? 


o Evaluate progress and make adjustments: Does the evaluation plan address the process and 
impact of implementing improvement strategies? 
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Developing the Evaluation Plan 


This section of the process guide will cover the development of the evaluation plan. A good evaluation plan 
is a written document that: 

e spells out exactly what the state is trying to accomplish, including the impact of the SSIP activities on 
the SIMR and other key outcomes such as changes to infrastructure and practice (i.e., intended 
short-term, intermediate, and long-term outcomes at each level of the system relative to strategies, 
activities, timelines and ranked priorities); 


outlines how the state will measure the intended outcomes and identifies the best way to capture the 
most relevant information (i.e., meaningful performance indicators, targeted evaluation questions, and 
specific data collection methods) that will enable staff to: 

a. determine whether activities are on the right track; 


b. make mid-course corrections to improve the implementation of activities; and 


c. determine the degree to which intended outcomes were achieved; 


provides opportunities to critically examine the extent to which implemented activities and strategies 
are functioning as intended and may guide mid-course corrections to implementation processes and 
activities; and 


provides opportunities for reporting and dissemination that address the question of whether or not what 
has been accomplished is what was intended. 


Stakeholder involvement is critical to all aspects of the SSIP process. Accordingly, state leaders need to 
use purposeful strategies to meaningfully engage those stakeholders. Evaluation is no exception. One way 
to ensure the relevance and usefulness of an evaluation is to include the perspectives and insights of as 
many individuals, groups, and other stakeholders as possible throughout the evaluation planning process. 
Engaging a wide range of stakeholders in each step of the evaluation planning process also provides 
opportunities to address questions, explore assumptions, and develop a shared understanding of what the 
evaluation will address and the findings it is expected to produce. Indeed, meaningful engagement of 
stakeholders can take a variety of forms. Sometimes stakeholder involvement is limited to reviewing or 
reacting to a draft of the evaluation plan, while in other cases, it is actually developing the evaluation plan 
alongside states. 
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Many resources exist related to evaluation. IDC developed A Guide to SSIP Evaluation Planning which 
includes worksheets to guide users through the evaluation process. Three other key resources that TA staff 
found useful in understanding all steps of evaluation were: 


The Program Manager's Guide to Evaluation, 2" Edition 
We Did It Ourselves: An Evaluation Guidebook 
Taking Stock: A Practical Guide to Evaluating Your Own Programs 


Evaluations can be presented in many different ways, with varying terms and components. The following 
steps show one way to conceptualize the steps involved in developing an evaluation plan. 


Step 1. Identify intended outcomes of key activities. 


Purpose Resources 
e Clarify and create a written record of the State developed SSIP Theory of Action 
expected results of activities and steps if developed in Phase | 


they are well implemented. State's completed SSIP Action Plan Template 
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Step 2. 


Step 3. 


Develop a logic model, periodically review and revise as needed. 


Purpose 


Link the SSIP theory of action developed in 


Phase | to the improvement plan completed 
in Phase Il. 


Create a visual representation of the logical 
relationships among activities, intended 
outcomes, and the SIMR. 


Identify gaps in logical relationships 
between the intended outcomes and SIMR. 


Ensure that the outcomes identified are 
sufficient to achieve the SIMR. 


Give priority to those outcomes most critical 
to achieving the SIMR. 


Include intended outcomes necessary to fill 
logical gaps. 


Eliminate outcomes not necessary to 
achieve the SIMR. 


Develop evaluation questions. 


Purpose 


Create a written record of the questions the 
state wants to answer with the evaluation. 


Align evaluation activities and state 
priorities. 


Resources 


State developed SSIP Theory of Action 


developed in Phase | 


W.K. Kellogg Foundation Logic Model 
Development Guide 


Logic Models as a Platform for Program 
Evaluation Planning, Implementation, and 
Use of Findings 


Developing a Logic Model 


Resources 


The Data are in the Details: Translating 
Evaluation Questions into Detailed 
Analytical Questions 


Critical Questions About Early Intervention and 
Early Childhood Special Education 


Guidance Table for Analyzing Child Outcomes 
Data for Program Improvement 
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Step 4. Develop performance indicators. 


Purpose 


e Create a written record of the metrics and 
criteria that will be used to define 
achievement of intended outcomes. 


e Ensure that evaluation questions can be 
answered with data collected. 


Step 5. Identify data source(s)/methods. 


Purpose 


e Create a written record of: 
© data collection methods, 
© tools/instruments for collecting data, 
© participants in evaluation, and 


o Existing data sources. 


e Ensure data will be available to answer the 


evaluation questions identified. 


Step 6. Plan preliminary data analysis. 


Purpose 


e Ensure that data collected are sufficient to 
answer evaluation questions. 


e Improved communication with staff 
responsible for data analysis. 


Resources 


State developed data dictionaries 


State developed activity timelines 


Resources 


State's Data Dicionary 


DaSy Data System Framework, Quality Indicator 
SD4 


Common Education Data Standards (CEDS) 


Resources 


Guidance Table for Analyzing Child Outcomes 
Data for Program Improvement 


Planning, Conducting, and Documenting Data 
Analysis for Program Improvement 


SSIP Phase II Process Guide (updated March 7, 2016) 


23 


Step 7. Develop timelines for the evaluation. 


Purpose Resources 
e Align evaluation activities with the overall State developed timelines from the SSIP 
timeline of improvement activities. improvement plan 


e Ensure the evaluation activities will provide 
information in a timely way. 


Step 8. Identify how the data will be used to inform implementation. 


Purpose Resources - 
e Ensure that data collected are sufficient to Al Hub Module 1, Framework 5: Improvement 
answer evaluation questions. Cycles 
e Improved communication with staff Al Hub Handout 14: Improvement Cycles 


responsible for data analysis. 


Step 9. Involve stakeholders the evaluation planning process. 


Purpose Resources 
e Identify stakeholders to be included in the A Practical Guide for Engaging Stakeholders in 
evaluation planning process. Developing Evaluation Questions 
e Prepare stakeholders to support Leading by Convening: A Blueprint for Authentic 
development and implementation of the Engagement 


eyaluavon' plan: Stakeholder Engagement in Data System 


e Ensure that stakeholders are included in Initiatives: An Online Module for Part C and 
each step, as appropriate. Part B 619 State Staff 
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Communicating the Plan 


There are many times throughout the development of an improvement plan when state staff should 
consider processes for effective communication. Such processes help to ensure that those developing and 
implementing the improvement and evaluation plans and those impacted by the plans can communicate 
easily. Appropriate and timely communication also allows for refinements to the plan throughout 
development and implementation, ultimately ensuring the success and sustainability of the activities. 


This section outlines steps, purposes, and resources in two subsections: 
e Development of communication activities within the improvement plan; and 


e Communication of completed Phase II plans. 


The "plan" referenced throughout includes both the improvement plan and the evaluation plan components 
of the overall SSIP. 
Key considerations for communication activities: 


e Do activities reflect communication in all directions (i.e., activities supporting communication from 
multiple stakeholders to the state and vice versa)? 


e Has sufficient time been allocated to request, gather, synthesize, and use input for each activity? 
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Step 1. Establish and document communication protocols for the SSIP team. 


Purpose 


e Establish a transparent communication 
process. 


e Provide clear expectations and 
responsibilities for team members. 


e Establish how the team communicates 
internally, with external stakeholders, and 
with the broader public. 


e Document how communication is facilitated 
and how frequently it should occur. 


Resources 


Al Hub Handout 8: Communication Protocol 
Worksheet 


Leading by Convening: One-Way and Two-Way 
Learning 


Step 2. Document the mission and purpose of the communication with external stakeholders. 


Purpose 


e Define the goals and reasons for 
communicating SSIP activities. 


Resources 


Template for Strategic Communications Plan 


Step 3. Identify stakeholder groups that need to be involved in each improvement strategy. 


Purpose 


e Ensure that the SSIP process will have input 
and feedback from diverse partners. 


e Document stakeholder involvement for 
OSEP. 


Resources 


Community Tool Box: Chapter 7, Section 8: 
Identifying and Analyzing Stakeholders and 
Their Interests 


Leading by Convening: Ensuring Relevant 
Participation 


Leading by Convening: Measuring Progress 
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Step 4. Identify existing communication supports and resources within the state. 


Purpose Resources 


e Build upon existing resources. Existing state communications and online 
support. 


Step 5. Develop communication processes in order to share progress between groups and 
receive input from stakeholder groups and public audiences. 


Purpose Resources 
e Provide opportunities for feedback from Al Hub Module 5, Topic 3: Practice-Policy 
groups that haven’t been represented on Feedback Loops 
Sol workteams. Al Hub Activity 2.4: Mapping Feedback and Feed 
e Ensure stakeholders are responsible for Forward Pathways 


communicating progress and gathering 
feedback from the groups they represent (or 
other groups as assigned). 


e Integrate feedback into plans prior to 
submitting to OSEP. 
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Communicating Completed Plans 


When thinking about communicating the completed Phase II SSIP document, strategies must go beyond 
dissemination. Communication should incorporate continual feedback loops that include sharing and 
receiving information. 


The following steps describe the milestones of communicating completed improvement and evaluation 
plans. These steps, purposes, and resources for communicating completed plans support meaningful 
interaction with stakeholders and the development of a high-quality Phase II SSIP. 


Step 1. Plot the communication strategies used so far to communicate with stakeholders and 
the public. 


Purpose Resources 
e Become aware of the types and levels of eating by Convening: One-Way and Two-Way 
strategies that are used. Learning (see pg. 6-7) 


e Build upon existing resources. 


e Identify any biases in communication 
strategies. 


Step 2. With core stakeholders, plan and co-create the messages to be conveyed. 


Purpose Resources 
e Use a consistent message about what the Template for Strategic Communications Plan 
SSIP is, how the state is planning to roll it (see pg. 8-11) 


out and the role of stakeholders and the 
public. 


e Include core features, components and 
expected outcomes of any new practices or 
systems to be implemented. 
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Step 3. Strategize communication channels needed to reach intended audiences. 


Purpose Resources 


e Intentionally select communication channels Choosing the Right Communication Channel 
(e.g. print, online, presentations) in order to 
reach targeted stakeholders and public 


Leading by Convening: Ensuring Relevant 


Participation 
audiences. 


Step 4. Create a list of stakeholder groups/public that have not yet been engaged so that they 
have opportunities to provide feedback for the plans. 


Purpose Resources 


e Use existing stakeholders to identify whose Leading by Convening: Meet the Stakeholders 
input is missing. 


Step 5. Develop feedback loops for dissemination of completed plans and receipt of feedback. 


Purpose Resources 


e Provide opportunities for stakeholder groups Leading by Convening: Dialogue Guides 
and public audiences to interact with and Problem of Practice: Creating Active 
respond to the plans (e.g., encourage 


stakeholders and champions to engage the 


Engagement 


Al Hub Module 5, Topic 3: Practice-Policy 
Feedback Loops 


groups they represent). 


e Ensure that the feedback that is being st 
submitted is intentionally considered and Cofnmunlbalon Cycles 


integrated. 


e Create a process where stakeholders who 
have provided input can see that their input 
was considered and, if appropriate, how it 
was used. 
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Step 6. Develop a timeline for communication of the completed plan, including allowance for 
time needed for public feedback. 


Purpose Resources 


e Provide adequate time to communicate and How-To: Make a Gannt Chart in Excel 
receive feedback on the completed plan. 
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SSIP Phase II Tools and Resources 


The items below include the resources used in Phase II of the SSIP. The potential uses of each resource are 
provided. 
Al Hub Activity 2.4: Mapping Feedback and Feed Forward Pathways 
Integration of Implementation Drivers, including creating information/communication pathways, is 


a key facet of doing Active implementation. This mapping activity, which includes an 
Implementations Drivers diagram, will help improve and integrate feedback and feed forward 


processes. 


Retrieved from http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/AlModules-Activity- 


2-4a-MappingCommunicationPathways.pdf 


Al Hub Activity 5.2: PDSA Who am I? 
An activity that highlights the importance of PDSA cycles in data-based decision making and 
guides participants in reflections about team strengths and weaknesses. 
Retrieved from http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/AlModules-Activity- 


5-2-PDSAWhoAmlI.pdf 


Al Hub Handout 8: Communication Protocol Worksheet 
This Communication Protocol Worksheet can be used to promote system alignment and 
facilitate communication. 


Retrieved from http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/AlHub-Handout8- 


CommunicationProtocolWorksheet.pdf 


Al Hub Handout 14: Improvement Cycles 
Implementation teams employ improvement cycles in order to intentionally identify problems and 
solutions. As a result, practices improve and hospitable environments are developed to support 
more effective and efficient ways to work. The Plan, Do, Study, Act Cycle or PDSA Cycle 
underlies the different types of improvement cycles described in this active implementation 
framework (Deming, 1986). 
Retrieved from http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/AlHub-Handout14- 


ImprovementCycles.pdf 
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Al Hub Module 1, Framework 5: Improvement Cycles 


Framework 5 of this webpage outlines two different conceptualizations of improvement cycles: 
Plan-Do-Study-Act, and Practice-Policy. 


Retrieved from http://implementation.fpg.unc.edu/module-1/improvement-cycles 


Al Hub Module 3: Implementation Teams 


This module located on the Active Implementation Hub provides an introduction to 
implementation teams, including definitions, rationale, key functions, and best practices for 
establishing and maintaining these teams. 


Retrieved from http://implementation.fpg.unc.edu/module-3 


Al Hub Module 5, Topic 3: Practice-Policy Feedback Loops 


Practice-Policy Feedback Loops are PDSA cycles designed to provide organizational leaders 
and policy makers with information from the practice level about implementation barriers and 
successes so that a more aligned system can be developed. 


Retrieved from http://implementation.fpg.unc.edu/module-5/topic-3-practice-policy-feedback-loops 


Benchmarks of Quality for Home-Visiting Programs 


This worksheet provides benchmarks of a high-quality home-visiting program and a 
self-assessment tool. 


Retrieved from http://ectacenter.org/~pdfs/calls/2015/decrp-2015-02-11/Benchmarks_Home Visiting.pdf 


Choosing the Right Communication Channel 


Communication cycles are designed to intentionally address communication gaps as education 
systems work to improve student outcomes. 


Retrieved from https://www.melcrum.com/research/strategy-planning-tactics-intranets-digital-social-media/choosing-right- 


communication 
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Circles of Involvement 


This is an exercise developed to identify key partners for implementing system changes. The 
version linked here includes instructions for use in a specific public health initiative. The most 
useful part of the document may be the definition of the Circles of Involvement. Consideration of 
all of the various types of stakeholders identified in the Circles of Involvement and the benefit 
they will bring to the implementation of the SSIP can improve the quality of stakeholder 
involvement. This process links to the Implementation Science recommendation that members 
of implementation teams should represent different perspectives and range across multiple 
levels of the system. 


Retrieved from http://www.naccho.org/topics/infrastructure/community-health-assessment-and-improvement-planning 


/upload/http___ www-naccho-org_topics_infrastructure_mapp_framework_clearinghouse_loader.pdf 


Common Education Data Standards (CEDS) 


The Common Education Data Standards (CEDS) project supports state work to develop 
common data standards for a key set of education data elements which streamlines the 
exchange, comparison, and understanding of data within and across P-20W. States already 
implementing CEDS can use the standards to identify data elements available for analysis. For 
states building their data system infrastructure to support the SSIP evaluation, CEDS would be 
an important resource for planning data collections. 


The CEDS Alignment Tool is a Web-based tool that allows stakeholders to: 


© import or input their organizations’ data dictionaries, 


© compare (or "map") their data dictionaries (element names, definitions, and options sets) to 
CEDS, and; 
© compare their data dictionaries with those of other participating organizations. 


The CEDS Connect Too! allows stakeholders from varied educational organizations to identify 
policy questions and related data elements, define analytic approaches, calculate metrics and 
indicators, address reporting requirements, and accomplish many other data tasks. 


Retrieved from https://ceds.ed.gov 
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Communication Cycles 


Facilitators and barriers encountered in practice are rapidly (at least monthly) communicated to 


the highest level required for a solution. For example, ‘not enough time for competent coaches to 


support teachers learning to use new instruction practices' can be taken from the school to the 
district leadership. If district leadership cannot find a solution for some reason, the issue is not 
dropped — it is taken to the State Management Team. In this way, local issues can be resolved 
locally and more systemic issues can be resolved at a statewide level. 


Retrieved from http://sisep.fpg.unc.edu/news/sisep-enotes-may-2015 


Community Tool Box: Chapter 7, Section 8: Identifying and Analyzing Stakeholders and Their 
Interests 


Identifying stakeholders and their interests is an important aspect of the participatory process. 
Chapter 7 of the Community Tool Box goes into depth about the process of identifying and 
engaging stakeholders. 


Retrieved from http://ctb.ku.edu/en/table-of-contents/participation/encouraging-involvement/identify-stakeholders/main 


Community Tool Box: Chapter 8, Section 5: Developing an Action Plan 
The Community Tool Box provides tips and guidelines for developing a strategic plan and 


organizational structure, from defining a vision to bringing about real change. 


Retrieved from http://ctb.ku.edu/en/table-of-contents/structure/strategic-planning/develop-action-plans/main 


Critical Questions About Early Intervention and Early Childhood Special Education 


This tool is designed to help programs identify evaluation questions. The tool is a good starting 
point for states beginning to or working toward customizing their evaluation questions to align 
with their needs. Evaluation questions are divided into three outcome areas: 

1. child/family, 

2. practitioner, and; 

3. program/agency. 
Also helpful: Each question is identified as essential ("must-have") or aspirational ("nice- 
to-have"). 


Retrieved from http://dasycenter.org/critical-questions 
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DaSy Data System Framework 


States can use this resource in planning the infrastructure to support their evaluation and data 
collection activities for the SSIP. The quality elements within each component outline what needs 
to be in place to collect and use data effectively. The DaSy Framework subcomponent Data 
Governance and Management outlines activities that are helpful when looking for secondary 
data and developing procedures for collecting new data. Section 2 of this subcomponent (Quality 
and Integrity) should be carefully considered when new data are being collected. Sections 1 
(Authority and Accountability) and 3 (Security and Access) must also be considered as states 
think about where new data will be housed and how the data can be accessed. 


Retrieved from: http://dasycenter.org/resources/dasy-framework 


The Data are in the Details: Translating Evaluation Questions into Detailed Analytical 
Questions 


This 2015 presentation focuses on breaking down and/or rephrasing evaluation questions, 
perhaps into several questions, that clearly communicate the details needed for data analysts to 
understand what you want to analyze and how. Clearly articulated analytical questions will help 
you to determine what data are needed and to more efficiently and effectively obtain the answers 
that you need, using data to support success in the State Systemic Improvement Plan (SSIP). 


Retrieved from http://dasycenter.org/the-data-are-in-the-details-albuquerque 


DEC Recommended Practices & Glossary 


The DEC Recommended Practices provide guidance to practitioners and families about the 
most effective ways to improve the learning outcomes and promote the development of young 
children, birth through age 5, who have or are at risk for developmental delays or disabilities. 


Retrieved from http://www.dec-sped.org/recommendedpractices 


Developing a High Quality Improvement Plan 


This presentation includes information about how VA Part C elicited membership for planning 
teams, how the team meetings are conducted, and how the teams are coordinated. This can be 
used as a model for developing a process. 


Kicking off Phase Il: Slides 11-23 are most relevant to staffing for planning. 
Developing the Improvement Plan: Slides 3-11 are most relevant. 


Retrieved from https://appam.certain.com/accounts/register123/air/events/pdconf/userfiles 


/0x1255388f65dDeveloping_a_High-Quality_Improv.pptx 
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Developing a Logic Model 


This is a teaching guide that can be used in the development of a logic model. It includes 
definitions of the components of a logic model (inputs, activities, outputs, outcomes) and 
examples of some of the categories underneath the components. 


University of Wisconsin-Extension. (2010a). Developing a logic model [Slide presentation]. Madison, WI: Author. 


Retrieved from http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html 


DIY Committee Guide: Strategic Plan Step 5: Writing Your Plan 


This guide gives a series of steps to help an organization plan its development strategically and 
outline its goals and aims. 


Retrieved from http://www.diycommitteeguide.org/resource/strategic-plan-step-5-writing-your-plan 


Getting Ready for Phase Il of the SSIP 


This annotated presentation provides an overview of Phase II focused on getting ready for 
Phase II of the SSIP. States can copy the slides into presentations that they are planning to kick 
off Phase Il. 


Retrieved from http://ectacenter.org/~ppts/calls/2014/ssip/ssip2-1-2014-12-01.pptx 


Guidance Table for Analyzing Child Outcomes Data for Program Improvement 


This guidance table is designed to help identify key issues, questions, and approaches for 
analyzing and interpreting data on outcomes for young children with disabilities. The tool outlines 
a series of steps related to defining analysis questions, clarifying expectations, analyzing data, 
testing inferences, and conducting data-based program improvement planning. It also includes 
examples of questions and approaches and sample figures to consider. 


Retrieved from http://ectacenter.org/eco/assets/pdfs/AnalyzingChildOutcomesData-GuidanceTable.pdf 


A Guide to the Implementation Process: Stages, Steps, and Activities: Stage 2: Installation 


This interactive guide on the implementation stages includes information on forming a state 
leadership team to plan and oversee implementation and scaling up of evidence-based 
practices. Content is provided on the development of the state leadership teams that links to the 
activities included in kicking off Phase Il. 


Retrieved from http://ectacenter.org/implementprocess/interactive/stage2/intro.asp 
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A Guide to SSIP Evaluation Planning 


This guide describes key steps for developing a well-thought-out plan for evaluating a SSIP. The 
guide provides considerations for how to incorporate each step into an evaluation plan, as well 
as a series of worksheets that correspond to each step and can be used to facilitate the planning 
process. The guide, along with its corresponding worksheets, is intended for TA providers to use 
in partnership with state staff. 


https://ideadata.org/resource-library/5697cca3140ba0ca5c8b4599 


The Hexagon Tool 


The Hexagon Tool helps states, districts, and schools systematically evaluate new and existing 
interventions via six broad factors: needs, fit, resource availability, evidence, readiness for 
replication and capacity to implement. 


Retrieved from http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/NIRN-Education- 


TheHexagonTool.pdf 


How-To: Make a Gannt Chart in Excel 


Although Excel is not a graphical tool by design, it is possible, with quite a bit of formatting, to 
make a Gantt chart in Excel. You can do this by turning your project tables into an Excel Gantt 
chart using Excel's bar graph functionality, and importing your Gantt charts into PowerPoint. 


Retrieved from https://www.officetimeline.com/gantt-chart-excel 


An Integrated Stage-Based Framework for Implementation of Early Childhood Programs and 
Systems 


This brief introduces key elements of effective implementation within an integrated, stage-based 
framework. This framework posits that (1) implementation happens in four discernible stages; 
and (2) three common "threads" or core elements exist across each of these stages. 


Kicking off Phase Il: Pages 5 and 6 include a useful summary of implementation teams. 
Included in the summary are: a definition of implementation teams, membership of 
implementation teams, relationships among teams, rational for using implementation team, and 
core competencies required of implementation teams. 


Developing the Improvement Plan: The entire document is useful for this activity. 


Retrieved from http://www.acf.hhs.gov/sites/default/files/opre/es_cceepra_stage_based_framework_brief_508.pdf 
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Leading by Convening: A Blueprint for Authentic Engagement 


Dialogue is necessary when implementing practices. This dialogue guide can be used to 
develop shared meaning and create effective dialogue. 


Retrieved from http://www.ideapartnership.org/building-connections/the-partnership-way.html 


Leading by Convening: Dialogue Guides 
This document contains models for stakeholder interaction around issues. 


Retrieved from http://www.ideapartnership.org/documents/NovUploads/Blueprint%20USB/Meetings%20to%20Co-Create 
/Dialogue%20Guides.pdf 


Leading by Convening: Ensuring Relevant Participation 


This document has tips for encouraging and supporting relevant participation from the full range 
of stakeholders, including asking at which level individuals want to be engaged early in your 
collaboration. 


Retrieved from http://www.ideapartnership.org/documents/NovUploads/Blueprint%20USB/Ensuring%20Relevant 
/Engage%20Everybody.pdf 


Leading by Convening: Measuring Progress 


These qualitative rubrics are tools that can be used to generate data on interactions. These data 
will help you understand the growth of critical relationships to ensure stakeholders are engaged. 


Retrieved from http://www.ideapartnership.org/documents/NovUploads/Blueprint%20USB/Bringing%20it%20All 
/Measuring%20Progress.pdf 


Leading by Convening: Meet the Stakeholders 


For every issue, there are a number of groups that have deep and durable connections at the 
practice level. Some are very closely aligned with the issues that you are trying to influence. 
Others have more distant, yet still important, connections. In either case, stakeholder groups 
have influence in what practitioners know, believe and do. Stakeholder groups can be important 
allies in moving new and/or proven practices to implementation! This template will help leaders 
identify and reach out to potential partners in order to meet and address persistent challenges. 


Retrieved from http://www.ideapartnership.org/documents/NovUploads/Blueprint%20USB/Coalescing%20Around 
/Meet%20the%20Stakeholders.pdf 
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Leading by Convening: One-Way and Two-Way Learning 


This PowerPoint presentation offers a blueprint for authentic engagement by considering 
different types of learning strategies, communication styles, and leadership models. 


Retrieved from http://www.ideapartnership.org/documents/NovUploads/Blueprint%20USB/Doing%20the%20Work 
/One-way%20and%20Two-way%20Learning.pdf 


Logic Models as a Platform for Program Evaluation Planning, Implementation, and Use of 
Findings 


This presentation is geared toward helping practitioners develop useful logic models. The slides 
address the hallmarks of well-constructed, useful logic models and how to use logic models for 
program evaluation planning, implementation, and using the findings. The slides break down and 
explain the different components and terminology associated with a logic model. Additional 
resources and sites are also cited. States can use this presentation as an introduction to logic 
models and as guidance for key components to create a meaningful logic model for their SSIP 
evaluation. 


Honeycutt, S. & Kegler, M.C. (2010). Retrieved from http://comm.eval.org/viewdocument/? DocumentKey=79c7da3d- 
6978-452c-8e8b-7056d3626966 


Part B SPP/APR Indicator Measurement Table 
The table describes the requirements for Phase II of the SSIP for Part B. 


Retrieved from https://osep.grads360.org/#communities/pdc/documents/4603 


Part B State Systemic Improvement Plan (SSIP) Phase Il OSEP Guidance and Review Tool 


The Part B SSIP Phase II OSEP Guidance and Review Tool is based on the three components 
described in Phase II of the Measurement Table under Indicator 17 (Part B). Those components 
are 1) Infrastructure Development; 2) Support for LEA Implementation of EBPs; and 3) 
Evaluation. Phase II builds on the five components developed in Phase |. Phase II must be 
submitted by April 1, 2016 as part of the FFY 2014 SPP/APR. The Phase II components are in 
addition to Phase | content (including any updates). 


Retrieved from https://osep.grads360.org/#communities/pdc/documents/8823 


Part C SPP/APR Indicator Measurement Table 
The table describes the requirements for Phase II of the SSIP for Part C. 


Retrieved from https://osep.grads360.org/#communities/pdc/documents/4604 
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Part C State Systemic Improvement Plan (SSIP) Phase Il OSEP Guidance and Review Tool 


The Part C SSIP Phase Il OSEP Guidance and Review Tool is based on the three components 
described in Phase II of the Measurement Table under Indicator 11 (Part C). Those components 
are 1) Infrastructure Development; 2) Support for EIS Programs and EIS Provider 
Implementation of EBPs; and 3) Evaluation. Phase II builds on the five components developed in 
Phase |. Phase II must be submitted by April 1, 2016 with the FFY 2014 SPP/APR. The Phase II 
components are in addition to Phase | content (including any updates). 


Retrieved from https://osep.grads360.org/#communities/pdc/documents/8824 


Planning, Conducting, and Documenting Data Analysis for Program Improvement 


This document provides information to help state staff define and limit the scope of data analysis 
for program improvement efforts, including the State Systemic Improvement Plan (SSIP); 
develop a plan for data analysis; document alternative hypotheses and additional analyses as 
they are generated; and summarize findings and document results. 


Retrieved from http://dasycenter.sri.com/downloads/DaSy_papers 


/DaSy_SSIP_DataAnalysisPlanning_20150323_FINAL_Acc.pdf 


Planning Guide to Statewide Implementation, Scale-up, and Sustainability of Recommended 
Practices 
This guide includes TA process documents and the implementation process, structures, and 
tools for planning and monitoring implementation used in ECTA Intensive TA for Implementing, 
Sustaining, & Scaling Up Evidence-Based Practices to Improve Child Outcomes. 


Kicking off Phase Il: The section on p. 6 provides an overview of the major structures of the 
RP2: Reaching Potentials through Recommended Practices Initiatives including: 

o State Leadership Team 

o Master Cadre of Training and TA providers 

°o Implementation and Demonstration Sites 
Also useful are the sections related to the Planning/Installation Stage embedded under each of 


the major structures. 


Developing the Improvement Plan: The entire document relates to developing the 
improvement plan. 


Retrieved from http://ectacenter.org/~pdfs/implement_ebp/ECTA_RP_StateGuide_2-2015.pdf 


SSIP Phase II Process Guide (updated March 7, 2016) 40 


A Practical Guide for Engaging Stakeholders in Developing Evaluation Questions 


This guide provides strategies for involving stakeholders in setting the direction for evaluation. 
While it is geared toward evaluation consultants, it provides useful information for state staff to 
guide efforts to promote stakeholder engagement throughout the SSIP process. 


Preskill, H., & Jones, N. (2009). Robert Wood Johnson Foundation Evaluation Series. Retrieved from http://www. rwif.org 


/content/dam/web-assets/2009/01/a-practical-guide-for-engaging-stakeholders-in-developing-evalua 


The Program Manager's Guide to Evaluation, 24 Edition 


This manual, written for program managers, contains tips, worksheets, and samples to help 
program managers understand each step of the evaluation process. 


Most relevant sections: 


© Outline of the basic evaluation steps (pg. 8-9) 

o Chapter 5: Outlines the steps to prepare for an evaluation (pg. 30-41) 

o Sample logic model and logic model worksheet (pg. 42-43) 

o Sample and worksheet for describing implementation objectives in measurable terms (pg. 
44-45) 

o Sample and worksheet for describing participant outcome objectives (pg. 46—47) 

o Sample outline for an evaluation plan (pg. 59-67) 

© Sample data collection plan (pg. 74-75) 

o Worksheet for developing a data collection plan (pg. 76) 


Retrieved from http://www.acf.hhs.gov/programs/opre/research/project/the-program-managers-guide-to-evaluation 


Problem of Practice: Creating Active Engagement 


This rubric describes the key actions and behaviors that leaders need to pay attention to in order 
to create active engagement at the informing level, networking level, collaborating level, and 
transforming level. 


Retrieved from https://wested.app.box.com/s/ohOwkij7a0a7hsfyfoxrvOf0u55ljdj3 


Recommended Resources for Planning to Evaluate Program Improvement Efforts (including 
the SSIP) 


This document provides a list of recommended resources to support evaluation planning for 
program improvement efforts, including the State Systemic Improvement Plan (SSIP). 


Retrieved from http://ectacenter.org/topics/ssip/plan_eval_program_improvement.asp 
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Sample SSIP Action Plan Template 


This sample action plan template provides states with a suggested, but not required, format and 
examples of potential content to assist them in completing their Phase II SSIP improvement plan 
and evaluation plan. 


Retrieved from http://ectacenter.org/~docs/topics/ssip/ssip_improvement_plan_template.doc 


SSIP Phase II Gantt Chart Template 
The Gantt Chart is a tool to track state planning activities with timelines. 


Retrieved from http://ectacenter.org/~docs/topics/ssip/ssip_phase_ii_gantt_chart.docx 


Stakeholder Engagement in Data System Initiatives: An Online Module for Part C and Part B 
619 State Staff 


This online module provides foundational information on effective stakeholder engagement and 
includes a session (Session 3) on strategies and resources for state staff to use when leading 
stakeholders in data system initiatives (including the SSIP). 


DaSy (2014). Retrieved from http://dasycenter.org/stakeholder-engagement-in-data-system-initiatives-an-online-module- 


for-part-c-and-part-b-61 9-state-staff-2 


State Leadership Team Benchmarks of Quality 


The State Benchmarks of Quality is can be used by the State Leadership Team (SLT) to assess 
progress and plan future actions so that Recommended Practices (RPs) are available for 
providers and families statewide. Sections of this document are particularly useful in considering 
the structure and staffing of the state leadership team, including on pages 3-4. 


Retrieved from http://ectacenter.org/~pdfs/implement_ebp/ECTA_RP_Benchmarks_4-2015.pdf 


A System Framework for Building High-Quality Early Intervention and Preschool Special 
Education Programs 


The system framework guides coordinators and staff in successfully addressing state needs, 
then implementing evidence-based practices, and finally bringing about positive outcomes for 
children and their families. 


Retrieved from http://ectacenter.org/sysframe 
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Taking Stock: A Practical Guide to Evaluating Your Own Programs 


This manual is written for community-based organizations and focuses on internal evaluation 
conducted by program staff, which will be useful for states planning to conduct their SSIP 
evaluation internally. The manual provides an overview of the evaluation process and includes 
the basic steps of planning for and conducting internal program evaluation, including practical 
strategies for identifying quantitative and qualitative data. 

Most relevant sections: 


o Chapter 4: What Are You Trying to Do? Defining Goals and Objectives (pg. 15—19) 
o Evaluation Planning Chart (pg. 25) 
o Chapter 6: Finding the Evidence: Strategies for Data Collection (pg. 27-37) 
© Chart of program objectives to evaluation questions (pg. 47) 
o Roadmap for evaluation design (pg. 61-62) 
o Appendices: Example evaluation reports 
Bond, S.L., Boyd, S.E., Rapp, K.A., Raphael, J.B. and Sizemore, B.A. (1997). Horizon Research. Retrieved from 


http://www.horizon-research.com/taking-stock-a-practical-guide-to-evaluating-your-own-programs 


Template for Strategic Communications Plan 


This template provides guidance for the development of a strategic communication plan (see 
Step 3: Develop Messages, pg. 2). 


Using the Message Development Worksheet will help convey goals and objectives, deliver 
important information about the issue, and compel the targeted audience to think, feel, or act. 
(see Message Worksheet, pg. 11) 


Retrieved from https://www.wkkf.org/resource-directory/resource/2006/01/template-for-strategic-communications-plan 


Vision and Direction in Leadership Checklist 


This checklist includes examples of steps leaders can take to help create a well-functioning and 
forward-thinking organization and to help practitioners feel a sense of belonging as they 
understand their purpose within the organization. The checklist can also be used as a 
self-evaluation by leaders at both state and local levels. 


Retrieved from http://ectacenter.org/~pdfs/decrp/LDR-3_Leaders_vision_direction.pdf 
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W.K. Kellogg Foundation Logic Model Development Guide 


This guide is written for a broad audience to clarify the what, why, and how of logic models. The 
contents of discussion include what logic models are and the different types, why a program 
should develop a logic model, and how a logic model can be used to guide implementation and 
plan for evaluation. 


The guide also includes templates and checklists that states can apply to their SSIP. This guide 
provides useful explanations and definitions of evaluation terminology. 


Most relevant sections: 


o Figure 2: How to read a logic model graphic (pg. 3) 
© 3 Approaches to logic models (pg. 9-10) 
Retrieved from https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model- 


development-guide 


We Did It Ourselves: An Evaluation Guidebook 


This guidebook is written for the person who has never done an evaluation before. It provides 
step-by-step instructions on how to design and carry out an evaluation. It could also be used as 
a reference by people interested in certain phases of the evaluation process, such as writing 
performance indicators or designing a survey. The guidebook details how to: 


1. develop outcome statements, indicators, and evaluation questions; 
2. formulate an evaluation methodology and collect, assess, and summarize data; and 
3. develop and disseminate evaluation findings and recommendations. 


The guidebook also contains a glossary, sample outcome and indicator statements, evaluation 
resources, and real-life stories of how community organizations used evaluation tools. This 
resource is lengthy but full of useful explanations of evaluation concepts. It has some wonderful 
worksheets and resources that states can use for the SSIP and that TA providers can use with 
states during sessions and workshops. 


Most relevant sections: 


o Worksheets to develop evaluation questions (pg. 35-39) 

o Implementation questions (pg. 58) 

o Worksheet for documenting strategies and activities (pg. 67) 

o Examples of documentation forms (pg. 62-66) 

o Examples and worksheets for developing an evaluation work plan (pg. 91-96) 
o Exercise for analyzing training attendance data (pg. 154-155) 


SRI International (supported by the Sierra Health Foundation) (2000). Retrieved from https://www.sierrahealth.org 
/pages/525 


SSIP Phase II Process Guide (updated March 7, 2016) 44 


WestEd B 


DaSy’ ECTACenter HDC H2N8 CSI 


The Center for IDEA 
Early Childhood Data Systems 


national center for systemic improvement 


SSIP Phase Il: Key Terms 


Note: A full glossary of terms from Implementation Science prepared by the ECTA Center is available at: 


http://ectacenter.org/implementprocess/glossary.asp 


Evaluation Plan: A written document describing how information will be collected about and used to 
inform key activities of the SSIP. 


Evaluation Questions: The key questions the state wants to answer with the evaluation. For example, 
are providers implementing the evidence based practices effectively? 


Evidence-Based Practices: "...for the early childhood field: Evidence-based practice is a decision- 
making process that integrates the best available research evidence with family and professional 
wisdom and values." (Buysse & Wesley, p.12) and “evidence-based practices can be defined as: 
Practices that are informed by research, in which the characteristics and consequences of 
environmental variables are empirically established and the relationship directly informs what a 
practitioner can do to produce a desired outcome." (Dunst, et al., p.3) 

Buysse, V., & Wesley, P. W. (2006). Evidence-based practice in the early childhood field. Washington, DC: ZERO 
TO THREE. https://secure2.convio.net/zttcfn/site/Ecommerce/193252082?VIEW_PRODUCT=true& 
product_id=1221&store_id=1461 


Dunst, C. J., Trivette, C. M., & Cutspec, P. A. (2007). Toward an operational definition of evidence-based practice. 
(Winterberry Research Perspectives, v.1, n.1). Morganton, NC: Winterberry Press. http://www.wbpress.com 


/shop/toward-an-operational-definition-of-evidence-based-practice/ 


Early Childhood Technical Assistance Center. (2015). System Framework for Part C & Section 619: Glossary of 


Terms. Retrieved from http://ectacenter.org/sysframe/glossary.asp#Evidence-Based_Practices 


Early Childhood Technical Assistance Center. (2015). Evidence-Based Practice Topical Webpage. Retrieved from 


http://ectacenter.org/topics/evbased/evbased.asp 


Feedback Loops: Feedback loops are communication processes used to gain input, analyze data and 
problem solve during the implementation process. Feedback loops are used among the State 
Leadership Team, Implementation Teams and Implementation Sites. 

Early Childhood Technical Assistance Center. (2014). Implementation Process: Glossary Terms. Retrieved from 


http://ectacenter.org/implementprocess/glossary.asp#def-feedbackloops 
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Implementation Drivers: Implementation drivers are a framework for organizing the capacity and 
infrastructure that influences the successful implementation of a new innovation or practice. Drivers 
include capacity for promoting competency through professional development, leadership and 
organizational supports such as policy and procedures, funding, administration, data systems, etc. 

Early Childhood Technical Assistance Center. (2014). Implementation Process: Glossary Terms. Retrieved from 


http://ectacenter.org/implementprocess/glossary.asp#def-implementationdrivers 


The Phase II plan should include the implementation drivers used to achieve short- and long-term 
SSIP outcomes. The National Implementation Research Network (NIRN) website provides detailed 
information about the types and uses of implementation drivers. 

The National Implementation Research Network. Implementation Drivers. Retrieved from http://nirn.fpg.unc.edu 


/learn-implementation/implementation-drivers 


Implementation Science: The principles of Implementation Science (Fixsen, et.a/., 2005) have been 
embedded into the design of the SSIP process and OSEP expects that states will use the principles 
in planning and implementing improvement strategies. The definition of the plan for Phase II from 
the Part C State Performance Plan (SPP) and Annual Performance Report (APR): Part C Indicator 
Measurement Table includes that the planning will be developed "with attention to the research on 
implementation." Throughout this document, we will introduce and embed key concepts of 
Implementation Science as they relate to the plan to be developed in Phase II. Included in each 
section are resources for readers to learn more about Implementation Science. Although all 
implementation frameworks (e.g. implementation teams, usable interventions, implementation 
stages, implementation drivers, and improvement cycles) need to be considered in Phase II, the 
implementation drivers are critical to address in the plan. 

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. &Wallace, F. (2005). Implementation Research: A 
Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health 


Institute, The National Implementation Research Network (FMHI Publication #231). 


Implementation Team: Implementation Teams (also called Local Leadership Teams) provide active 
leadership at the regional or program level to manage the implementation efforts and support the 
people using the new innovation or practice. The teams engage in continuous communication and 
feedback with the State Leadership Team about the issues, successes, and needed resources to 
support successful implementation and expansion. 

Early Childhood Technical Assistance Center. (2014). Implementation Process: Glossary Terms. Retrieved from 


http://ectacenter.org/implementprocess/glossary.asp#def-implementationteam 


Improvement Plan: A written document that includes the activities and steps for implementing the 
improvement strategies to achieve the intended outcomes. 


Improvement Strategies: A state's improvement strategies outline the course of action in achieving the 
Theory of Action. 
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Logic Model: A systematic and visual way to present and share your understanding of the relationships 
among the resources you have to operate your program, the activities you plan, and the changes or 
results you hope to achieve. 


W.K. Kellogg Foundation, 2004. W.K. Kellogg Foundation Logic Model Development Guide. Retrieved from 
https://www.wkkf.org/resource-directory/resource/2006/02/wk-kellogg-foundation-logic-model-development-guide 
Outcomes: 
Intended Outcome: Outcomes at all levels of the system (state, regional/local, practitioner, 
family, and child) that are intended to be achieved by implementing the specified 
improvement strategy to improve the state's SIMR (long-term outcome). States can use 


the "assumptions" from their Theory of Action (if sufficiently detailed) to identify their 
intended outcomes. 


Intermediate Outcome: Changes in actions or behaviors based on knowledge or skills 
acquired through outputs. 


Changes in adult actions or behaviors based on knowledge or skills acquired 
Fidelity of the intervention 

Improved organizational functioning 

Improved infrastructure and system functioning 


Long-Term Outcome: The results that fulfill the SSIP's goals; the SIMR is the key long-term 
outcome but some states may have others. 
=» Broadest program outcomes 


= Results that fulfill the project's goals 
= Impact on children or families 
=» Program scale-up and sustainability 


Example: [SIMR] There will be an increase in the percentage of infants and toddlers 
exiting early intervention services who demonstrate an increased rate of growth in positive 
social-emotional development. 


Short-Term Oucome: Direct results of the activities and their outputs. 
« What participants learn as a result of activities/outputs 


« What awareness, attitudes, or skills participants develop 


Example: El practitioners have improved understanding of child development including 
social-emotional development for infants and toddlers. 


PDSA (Plan, Do, Study, Act): An iterative, four-stage problem-solving model used for improving a 
process or carrying out change 
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Performance Indicator: The item of information that measures whether intended outcomes are being 
achieved. For example, an indicator might be: “An increase (direction) in the average score 
(number) on the Proficiency Test given at the end of training (method of measurement). 


Example: There will be an increase in the percent of providers who can correctly identify age 
appropriate social-emotional skills after the training. 


Stakeholder: An individual or group directly or indirectly affected by an initiative or project. 


Stakeholder Engagement: The use of stakeholders as participants in a collaborative process that 
guides the planning, implementation, and monitoring of an initiative or project. 


State Leadership Team: (also called state management or state implementation team) is the group of 
individuals at the state level who manage the change effort by actively leading and providing the 
internal supports needed to move the selected innovation or practice through all the stages and 
steps of implementation. 

Early Childhood Technical Assistance Center. (2014). Implementation Process: Glossary Terms. Retrieved from 


http://ectacenter.org/implementprocess/glossary.asp#def-stateleadershipteam 


Theory of Action (TOA): The TOA provides a general statement of the rationale for the state's 
improvement strategies. 


Terms of Reference: Terms of Reference describe a format for setting guidelines and expectations for 
team function, scope, and mission. A key part of a Terms of Reference document is to outline the 
communication protocols for a project. 


Usable Intervention: A usable intervention needs to be teachable, learnable, doable, and readily 
assessed in practice if it is to be used effectively to reach all students who could benefit. 
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Contributors to this SSIP Phase Il: Process Guide 


This SS/P Phase II Process Guide describes the requirements and a proposed process for developing a 


high-quality plan, and is intended to support states in designing a high-quality process for completing 
Phase II. The guide will be updated as content is developed and finalized. 


The Process Guide was developed by a collaborative team from ECTA, DaSy, NCSI and IDC 


including: 
Grace Kelley, ECTA/DaSy/ NCSI Anne Lucas, ECTA/DaSy 
Siobhan Colgan, IDC Cornelia Taylor, ECTA/DaSy/NCSI 
Carolee Eslinger, IDC Megan Vinh, ECTA/DaSy 


Monica Mathur-Kalluri, NCSI 


The following additional individuals provided input into the content of the Guide: 
Betsy Ayankoya, ECTA/DaSy Kristin Reedy, NCSI 


Jeanna Mullens, IDC Joanne Cashman, NCSI 


Linda Lynch, IDC 


The contents of this guide were developed under cooperative agreement numbers #H326R140006 
(DaSy), #H326P120002 (ECTA Center), #H373Y130002 (IDC) and #H326R140006 (NCSI) from the Office 
of Special Education Programs, U.S. Department of Education. Opinions expressed herein do not 
necessarily represent the policy of the US Department of Education, and you should not assume 
endorsement by the Federal Government. 
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